Uncoupled Analysis of Stochastic Reaction Networks in Fluctuating Environments

The dynamics of stochastic reaction networks within cells are inevitably modulated by factors considered extrinsic to the network such as for instance the fluctuations in ribsome copy numbers for a gene regulatory network. While several recent studies demonstrate the importance of accounting for such extrinsic components, the resulting models are typically hard to analyze. In this work we develop a general mathematical framework that allows to uncouple the network from its dynamic environment by incorporating only the environment's effect onto the network into a new model. More technically, we show how such fluctuating extrinsic components (e.g., chemical species) can be marginalized in order to obtain this decoupled model. We derive its corresponding process- and master equations and show how stochastic simulations can be performed. Using several case studies, we demonstrate the significance of the approach. For instance, we exemplarily formulate and solve a marginal master equation describing the protein translation and degradation in a fluctuating environment.Comment: 7 pages, 4 figures, Appendix attached as SI.pdf, under submissio

Moment-Based Variational Inference for Markov Jump Processes

We propose moment-based variational inference as a flexible framework for approximate smoothing of latent Markov jump processes. The main ingredient of our approach is to partition the set of all transitions of the latent process into classes. This allows to express the Kullback-Leibler divergence between the approximate and the exact posterior process in terms of a set of moment functions that arise naturally from the chosen partition. To illustrate possible choices of the partition, we consider special classes of jump processes that frequently occur in applications. We then extend the results to parameter inference and demonstrate the method on several examples.Comment: Accepted by the 36th International Conference on Machine Learning (ICML 2019

Marginal process framework: A model reduction tool for Markov jump processes

Markov jump process models have many applications across science. Often, these models are defined on a state-space of product form and only one of the components of the process is of direct interest. In this paper, we extend the marginal process framework, which provides a marginal description of the component of interest, to the case of fully coupled processes. We use entropic matching to obtain a finite-dimensional approximation of the filtering equation, which governs the transition rates of the marginal process. The resulting equations can be seen as a combination of two projection operations applied to the full master equation, so that we obtain a principled model reduction framework. We demonstrate the resulting reduced description on the totally asymmetric exclusion process. An important class of Markov jump processes are stochastic reaction networks, which have applications in chemical and biomolecular kinetics, ecological models and models of social networks. We obtain a particularly simple instantiation of the marginal process framework for mass-action systems by using product-Poisson distributions for the approximate solution of the filtering equation. We investigate the resulting approximate marginal process analytically and numerically.Comment: 16 pages, 5 figures; accepted for publication in Physical Review

A variational approach to path estimation and parameter inference of hidden diffusion processes

We consider a hidden Markov model, where the signal process, given by a diffusion, is only indirectly observed through some noisy measurements. The article develops a variational method for approximating the hidden states of the signal process given the full set of observations. This, in particular, leads to systematic approximations of the smoothing densities of the signal process. The paper then demonstrates how an efficient inference scheme, based on this variational approach to the approximation of the hidden states, can be designed to estimate the unknown parameters of stochastic differential equations. Two examples at the end illustrate the efficacy and the accuracy of the presented method.Comment: 37 pages, 2 figures, revise

Jump-Diffusion Approximation of Stochastic Reaction Dynamics: Error bounds and Algorithms

Biochemical reactions can happen on different time scales and also the abundance of species in these reactions can be very different from each other. Classical approaches, such as deterministic or stochastic approach, fail to account for or to exploit this multi-scale nature, respectively. In this paper, we propose a jump-diffusion approximation for multi-scale Markov jump processes that couples the two modeling approaches. An error bound of the proposed approximation is derived and used to partition the reactions into fast and slow sets, where the fast set is simulated by a stochastic differential equation and the slow set is modeled by a discrete chain. The error bound leads to a very efficient dynamic partitioning algorithm which has been implemented for several multi-scale reaction systems. The gain in computational efficiency is illustrated by a realistically sized model of a signal transduction cascade coupled to a gene expression dynamics.Comment: 32 pages, 7 figure

Under-approximating Cut Sets for Reachability in Large Scale Automata Networks

In the scope of discrete finite-state models of interacting components, we present a novel algorithm for identifying sets of local states of components whose activity is necessary for the reachability of a given local state. If all the local states from such a set are disabled in the model, the concerned reachability is impossible. Those sets are referred to as cut sets and are computed from a particular abstract causality structure, so-called Graph of Local Causality, inspired from previous work and generalised here to finite automata networks. The extracted sets of local states form an under-approximation of the complete minimal cut sets of the dynamics: there may exist smaller or additional cut sets for the given reachability. Applied to qualitative models of biological systems, such cut sets provide potential therapeutic targets that are proven to prevent molecules of interest to become active, up to the correctness of the model. Our new method makes tractable the formal analysis of very large scale networks, as illustrated by the computation of cut sets within a Boolean model of biological pathways interactions gathering more than 9000 components

Inverse Reinforcement Learning in Swarm Systems

Inverse reinforcement learning (IRL) has become a useful tool for learning behavioral models from demonstration data. However, IRL remains mostly unexplored for multi-agent systems. In this paper, we show how the principle of IRL can be extended to homogeneous large-scale problems, inspired by the collective swarming behavior of natural systems. In particular, we make the following contributions to the field: 1) We introduce the swarMDP framework, a sub-class of decentralized partially observable Markov decision processes endowed with a swarm characterization. 2) Exploiting the inherent homogeneity of this framework, we reduce the resulting multi-agent IRL problem to a single-agent one by proving that the agent-specific value functions in this model coincide. 3) To solve the corresponding control problem, we propose a novel heterogeneous learning scheme that is particularly tailored to the swarm setting. Results on two example systems demonstrate that our framework is able to produce meaningful local reward models from which we can replicate the observed global system dynamics.Comment: 9 pages, 8 figures; ### Version 2 ### version accepted at AAMAS 201

A Comprehensive Analysis of Swarming-based Live Streaming to Leverage Client Heterogeneity

Due to missing IP multicast support on an Internet scale, over-the-top media streams are delivered with the help of overlays as used by content delivery networks and their peer-to-peer (P2P) extensions. In this context, mesh/pull-based swarming plays an important role either as pure streaming approach or in combination with tree/push mechanisms. However, the impact of realistic client populations with heterogeneous resources is not yet fully understood. In this technical report, we contribute to closing this gap by mathematically analysing the most basic scheduling mechanisms latest deadline first (LDF) and earliest deadline first (EDF) in a continuous time Markov chain framework and combining them into a simple, yet powerful, mixed strategy to leverage inherent differences in client resources. The main contributions are twofold: (1) a mathematical framework for swarming on random graphs is proposed with a focus on LDF and EDF strategies in heterogeneous scenarios; (2) a mixed strategy, named SchedMix, is proposed that leverages peer heterogeneity. The proposed strategy, SchedMix is shown to outperform the other two strategies using different abstractions: a mean-field theoretic analysis of buffer probabilities, simulations of a stochastic model on random graphs, and a full-stack implementation of a P2P streaming system.Comment: Technical report and supplementary material to http://ieeexplore.ieee.org/document/7497234